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MEMORANDUM  FOR  AUDITOR  GENERAL,  DEPARTMENT  OF  THE  AIR  FORCE 

SUBJECT:  Quality  Control  Review  of  Air  Force  Audit  Agency  ’s  Special  Access 
Program  Audits  (Report  No.  D-2005-6-009) 

We  are  providing  this  report  for  your  information  and  use.  We  reviewed  the 
Air  Force  Audit  Agency  (AFAA)  system  of  quality  control  for  Special  Access  Program 
(SAP)  audits  for  the  three  years  ended  September  30,  2004.  The  Government  Auditing 
Standards  (GAS)  require  that  an  audit  organization  performing  audits  and/or  attestation 
engagements  in  accordance  with  GAS  should  have  an  appropriate  internal  quality  control 
system  in  place  and  undergo  an  external  peer  review  at  least  once  every  3  years  by 
reviewers  independent  of  the  audit  organization  being  reviewed.  As  the  organization  that 
has  audit  policy  and  oversight  responsibilities  for  audits  in  the  Department  of  Defense, 
we  conducted  this  external  peer  review  of  the  AFAA  SAP  audits  in  conjunction  with  the 
Naval  Audit  Service’s  external  peer  review  of  AFAA  non-SAP  audits. 

An  audit  organization’s  quality  control  policies  and  procedures  should  be 
appropriately  comprehensive  and  suitably  designed  to  provide  reasonable  assurance  of 
meeting  the  objectives  of  quality  control.  We  tested  the  AFAA  system  of  quality  control 
to  the  extent  considered  appropriate. 

In  our  opinion,  the  system  of  quality  control  for  the  audit  function  of  the  AFAA 
SAP  audits  in  effect  for  the  period  ended  September  30,  2004,  was  designed  in 
accordance  with  quality  standards  established  by  GAS.  Further,  the  internal  quality 
control  system  was  operating  effectively  to  provide  reasonable  assurance  that  SAP  audit 
personnel  were  following  established  policies,  procedures,  and  applicable  auditing 
standards.  Accordingly,  we  are  issuing  an  unqualified  opinion  on  your  SAP  audit  quality 
control  system  for  the  review  period  ended  September  30,  2004. 

Appendix  B  contains  comments,  observations,  and  recommendations  where 
AFAA  can  improve  its  quality  control  program  related  to  SAP  audits,  as  well  as  our 
responses  to  AFAA  management  comments  to  the  draft  report.  Appendix  C  provides  the 
full  text  of  management  comments  in  response  to  the  draft  report. 


See  Appendix  A  for  the  scope  and  methodology  of  the  review.  We  wish  to 
express  our  thanks  to  you  and  your  staff  for  your  cooperation  and  professionalism. 
Please  contact  Mr.  Robert  L.  Kienitz,  Acting  Project  Manager,  at  (703)  604-8754  or 
Ms.  Carolyn  R.  Davis,  Deputy  Assistant  Inspector  General  for  Audit  Policy  and 
Oversight  at  (703)  604-8877  if  you  have  any  questions. 

Patncia  A.  Brannin 
Assistant  Inspector  General 
Audit  Policy  and  Oversight 


Cc:  Director,  DoD  Special  Access  Program  Coordination  Office 
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Appendix  A.  Scope  and  Methodology 


We  limited  our  review  to  the  adequacy  of  AFAA  SAP  auditors’  compliance  with 
quality  policies,  procedures,  and  standards.  We  judgmentally  selected  3  SAP  audits  from 
a  universe  of  27  formal  reports  issued  by  AFAA  SAP  auditors  in  FYs  2002,  2003,  and 
2004,  and  tested  each  audit  for  compliance  with  the  AFAA  system  of  quality  control. 

The  Naval  Audit  Service  conducted  a  review  of  the  AFAA  internal  quality  control  system 
for  non-SAP  audits  and/or  attestation  engagements  and  is  issuing  a  separate  report.  The 
Assistant  Inspector  General  for  Audit  Policy  and  Oversight  will  issue  an  overall  opinion 
report  on  the  AFAA  internal  quality  control  system  that  will  include  the  combined  results 
of  the  reviews  of  SAP  and  non-SAP  audits. 

In  performing  our  review,  we  considered  the  requirements  of  quality  control  standards 
and  other  auditing  standards  contained  in  the  2003  Revision  of  the  Government  Auditing 
Standards  (GAS)  issued  by  the  Comptroller  General  of  the  United  States.  GAS  3.52 
states: 


The  external  peer  review  should  determine  whether,  during  the  period  under 
review,  the  reviewed  audit  organization’s  internal  quality  control  system  was 
adequate  and  whether  quality  control  policies  and  procedures  were  being 
complied  with  to  provide  the  audit  organization  with  reasonable  assurance  of 
conforming  to  applicable  professional  standards.  Audit  organizations  should  take 
remedial,  corrective  actions  based  on  the  results  of  the  peer  review. 

We  conducted  this  review  in  accordance  with  standards  and  guidelines  established  in  the 
Draft  2004  President’s  Council  on  Integrity  and  Efficiency  (PCIE)  “Guide  for 
Conducting  External  Peer  Reviews  of  the  Audit  Operations  of  Offices  of  Inspector 
General.”  We  modified  the  Guide  to  ensure  consistency  with  the  Naval  Audit  Service 
review  of  non-SAP  audits,  and  to  reflect  the  unique  nature  of  auditing  within  a  SAP 
environment.  We  interviewed  AFAA  auditors  and  their  managers,  reviewed  AFAA 
internal  audit-related  policies  and  procedures.  We  performed  this  review  in  May  2005  at 
three  AFAA  field  offices 

We  used  the  following  criteria  to  select  the  audits  under  review: 

•  Worked  backward  starting  with  FY  2004  audits  in  order  to  review  the  most 
current  quality  assurance  procedures  in  place. 

•  Eliminated  Base  Realignment  and  Closure  audits  because  they  are  not 
considered  typical  audits. 

•  Avoided  audits  with  multiple  SAPs  associated  with  the  audit  for  ease  of  access. 

•  Avoided  audits  that  have  the  same  or  similar  titles  to  ensure  review  of  multiple 
types  of  projects. 
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The  following  table  identifies  the  specific  reports  reviewed. 


Report  Number 

Date 

Title 

F2004-0004-FA0900 

26  Jan  2004 

“Cash  Controls” 

F2004-0005-FA0900 

4  Jun  2004 

“Financial  and  Contracting  Management” 

F2004-0007-FA0900 

16  Jul  2004 

“Support  Agreements” 

Limitations  of  Review.  Our  review  would  not  necessarily  disclose  all  weaknesses  in  the 
system  of  quality  control  or  all  instances  of  noncompliance  with  it  because  we  based  our 
review  on  selective  tests.  There  are  inherent  limitations  in  considering  the  potential 
effectiveness  of  any  quality  control  system.  In  performing  most  control  procedures, 
departures  can  result  from  misunderstanding  of  instructions,  mistakes  of  judgment, 
carelessness,  or  other  human  factors.  Projecting  any  evaluation  of  a  quality  control 
system  into  the  future  is  subject  to  the  risk  that  one  or  more  procedures  may  become 
inadequate  because  conditions  may  change  or  the  degree  of  compliance  with  procedures 
may  deteriorate. 
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Appendix  B.  Comments,  Observations,  and 

Recommendations 


We  are  issuing  an  unqualified  opinion  because  this  is  the  first  external  peer  review 
for  AFAA  SAP  audits  and  the  issues  we  identified  were  not  cumulatively  significant  to 
the  reports’  findings,  conclusions,  and  recommendations.  Overall,  we  found  that  AFAA 
could  improve  the  quality  control  system  and  guidance  for  SAP  audits  related  to  the  areas 
of  Independence,  Audit  Planning,  Supervision,  Evidence  and  Audit  Documentation, 
Reporting,  and  Quality  Assurance.  Implementing  the  recommendations  identified  below 
would  improve  the  quality  control  system  and  help  maintain  an  unqualified  opinion. 

Independence.  GAS  3.03  states  that  audit  organizations  and  individual  auditors  “should 
be  free  both  in  fact  and  appearance  from  personal,  external,  and  organizational 
impairments  to  independence.”  AFAA  Instruction  (AFAAI)  65-103,  “Audit 
Management  and  Administration,”  requires  auditors  to  sign  an  Independence  Statement 
at  the  beginning  of  each  audit  to  “certify  there  are  no  relationships  and  beliefs  that  might 
cause  the  auditor  or  supervisor  to  limit  the  extent  of  the  inquiry,  limit  disclosure,  or  slant 
audit  findings  in  any  way.” 

We  found  no  indications  of  impairment  during  our  review.  However,  there  were  no 
Independence  Statements  for  two  of  the  three  audits.  The  third  audit  had  an 
Independence  Statement  that  had  been  backdated  by  the  auditor,  who  used  a  version  of 
the  AFAA  Independence  Statement  that  was  not  in  effect  until  April  2004,  after  the  date 
of  his  signature  -  August  2003. 

The  AFAA  Chief,  Special  Programs  Division  stated  that  he  had  been  unaware  of  the 
requirement  for  project-specific  Independence  Statements  until  the  fall  of  2003.  He 
relied  on  annual  Financial  Disclosure  Statements  and  the  AFAA  Annual  Certification 
acknowledging  the  need  to  report  any  conflicts  of  interest  to  verify  the  auditors’ 
independence.  However,  AFAA  had  the  project-specific  Independence  Standard  in  place 
since  September  2002  and  all  three  projects  were  initiated  after  this  date.  AFAA  Policy, 
Oversight,  and  Systems  Division  personnel  stated  to  us  that  new  policy  is  placed  on  the 
AFAA  intranet  homepage,  and  disseminated  in  an  e-mail  to  all  AFAA  personnel. 
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Audit  Planning.  GAS  7.02  states  that  “work  is  to  be  adequately  planned,”  and 
GAS  7.07  states  that  “planning  should  be  documented.”  GAS  7.41  requires  auditors  to 
document  the  planning,  and  states: 

[T]he  form  and  content  of  the  written  audit  plan  will  vary  among  audits,  but 
should  include  an  audit  program  or  project  plan,  a  memorandum,  or  other 
appropriate  documentation  of  key  decisions  about  the  audit  objectives,  scope,  and 
methodology  and  of  the  auditors’  basis  for  those  decisions.  It  should  be  updated, 
as  necessary,  to  reflect  any  significant  changes  to  the  plan  made  during  the  audit. 

AFAAI  65-101,  “Internal  Audit  Procedures,”  contains  guidance  on  audit  planning  as  well 
as  a  template  “audit  planning  program”  that  all  AFAA  auditors  are  required  to  use  during 
audits.  AFAAI  65-101  states  that  audit  managers  should: 

...continuously  monitor  auditor  progress  during  the  planning  phase,  provide 
assistance  as  needed,  and  assure  the  auditor  conducts  the  planning  phase  in 
accordance  with  AFAA  policies  and  procedures;  and  ensure  the  auditor  uses  the 
Audit  Planning  Program  and  either  completes  each  step  or  provides  rationale  for 
not  completing  the  step. 

We  found  insufficient  documentation  related  to  audit  planning  in  all  three  AFAA  SAP 
audits  we  reviewed.  For  example,  one  of  the  audits  did  not  have  an  audit  program  and 
another  did  not  have  supervisory  approval  for  audit  steps  that  were  either  not  completed 
during  the  audit  or  were  marked  “N/A.” 

Supervision.  GAS  7.45  requires  staff  to  be  properly  supervised  and  GAS  7.47  states  that 
supervisory  reviews  of  audit  work  should  be  documented.  AFAAI  65-103  requires 
supervisors  to  use  an  “Audit  Review  Record”  to  document  the  supervision  and 
supervisory  reviews.  According  to  AFAAI  65-103,  the  review  record  must  include  the 
supervisor’s  initials  and  date,  and  the  working  papers  must  contain  “clear  evidence”  of 
supervisory  review.  We  found  that  several  working  papers  in  each  of  the  audits  were 
either  not  reviewed  in  a  timely  manner  or  not  reviewed  at  all  by  a  supervisor.  The  AFAA 
Chief,  Special  Programs  Division  stated  that  since  the  office  locations  are  widely 
scattered  across  the  U.S.,  he  visits  each  office  only  three  to  four  times  per  year.  In  two 
audits,  this  prevented  him  from  being  able  to  review  the  working  papers  until  five  months 
after  they  were  written.  This  is  mitigated  somewhat  by  the  fact  that  all  of  the  auditors  at 
the  AFAA  SAP  field  offices  are  senior  auditors. 

We  recognize  that  the  SAP  audit  environment  presents  challenges  for  complying  with 
some  of  the  AFAA  quality  processes.  However,  supervision  is  a  key  government 
auditing  standard  and  a  key  element  of  an  effective  quality  program.  Supervisors  must 
find  ways  to  provide  proper  supervision  even  within  such  an  environment. 
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Evidence  and  Audit  Documentation.  Working  papers  are  used  to  organize,  prepare, 
and  collect  relevant  evidence  and  documentation  during  an  audit.  GAS  7.66  requires  that 
auditors  prepare  and  maintain  audit  documentation,  and  that  the  audit  documentation 
should  contain  support  for  findings,  conclusions,  and  recommendations  before  auditors 
issue  their  report.  GAS  7.68  states  that  the  audit  documentation  forms  the  principal 
support  for  the  auditors’  report. 

AFAAI  65-103  requires  auditors  to  document  the  work  performed  to  support  significant 
conclusions  and  judgments,  including  descriptions  of  transactions  and  records  examined 
that  would  enable  an  experienced  auditor  to  examine  the  same  transactions  and  records. 
AFAAI  65-103  provides  the  following  guidelines  for  working  papers  (excerpt):  “The 
working  papers  should  be  understandable  without  supplementary  oral  explanations. 
Anyone  using  or  reviewing  the  working  papers  should  be  able  to  readily  determine  their 
purpose  and  source,  the  nature  and  scope  of  the  work  performed,  and  the  auditor’s 
conclusions.” 

Although  we  found  AFAA  SAP  audit  working  papers  contained  sufficient,  competent, 
and  relevant  evidence  to  support  judgments  and  conclusions  in  the  reports,  improvements 
could  be  made  to  audit  documentation.  We  found  that  6  of  the  15  judgmentally  selected 
facts  and  figures  we  reviewed  in  one  report,  10  of  15  in  another  report,  and  1  of  15  in  the 
third  report  could  have  been  better  supported.  Some  of  the  deficiencies  we  noted  during 
our  review  were: 

•  Source  documents  were  missing  information  about  the  provider  of  the  source 
documents,  and  in  some  cases  it  was  impossible  to  determine  whether  a  document 
was  auditor-generated  or  a  source  document. 

•  Several  figures  in  the  report  had  no  audit  trail  to  source  documents.  We  eventually 
identified  the  source  documents  and  verified  the  data  based  on  our  review  of  source 
documents. 

•  A  figure  in  the  report  referenced  back  to  a  note  written  by  the  auditor  that  did  not 
include  any  source  information.  During  our  review,  the  auditor  attributed  the 
statement  to  management,  but  there  was  no  attribution  on  the  note  or  in  the  report. 
With  such  attribution  a  reviewer  of  the  working  papers  or  of  the  report  can  not 
know  if  the  management  that  provided  the  information  was  a  reliable  source. 

•  Working  papers,  such  as  background  information  and  briefing  charts,  were  missing 
the  AFAA-required  elements  of  purpose,  source,  and  conclusion. 
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Reporting.  GAS  8.45  states  that  evidence  included  in  audit  reports  should 
demonstrate  the  correctness  and  reasonableness  of  the  matters  reported...”  GAS  provides 
a  best  practice  to  help  ensure  this  standard: 

One  way  to  help  ensure  the  audit  report  is  accurate  is  to  use  a  quality  control 
process  such  as  referencing...  in  which  an  experienced  auditor  who  is 
independent  of  the  audit  verifies  that  statements  of  fact,  figures,  and  dates  are 
correctly  reported,  that  findings  are  adequately  supported  by  the  audit 
documentation,  and  that  the  conclusions  and  recommendations  flow  logically 
from  the  support. 

AFAAI  65-101  requires  auditors  to  obtain  an  independent  reference  review  (IRR)  before 
the  draft  report  is  issued  to  management.  The  guidance  requires  the  independent 
referencers  to  trace  information  in  the  cross-referenced  draft  report  through  the  summary 
working  papers  to  the  supporting  working  papers  to  determine  that  they  are  consistent 
with  and  supported  by  the  working  papers  and  to  “be  alert  to  statements  in  the  report 
which  seem  illogical  or  lack  clarity,  ensure  the  team  chief  has  reviewed  all  supporting 
working  papers  and  cleared  any  review  notes,  and  refuse  to  sign  the  IRR  record  until  the 
team  chief  has  finished  reviewing  and  signed  off  on  the  working  papers.” 

We  found  that  the  three  AFAA  SAP  audit  reports  generally  met  the  GAS  reporting 
standards.  However,  none  of  the  SAP  reports  we  reviewed  complied  with  the  AFAA 
requirements  for  IRRs.  None  of  the  three  reports  received  an  IRR  before  the  draft  report 
was  issued  to  management,  and  two  of  the  three  reports  received  no  IRR.  The  AFAA 
Chief,  Special  Programs  Division  stated  that  he  was  unable  to  conduct  IRRs  before  the 
draft  reports  were  issued  due  to  a  lack  of  resources  within  the  SAP  environment,  and  that 
he  relied  on  the  experience  of  the  auditors  to  ensure  the  quality  of  the  audits. 

We  found  that  the  one  completed  IRR  was  not  adequately  documented;  specifically,  the 
referencer’s  signature  on  the  IRR  document  was  not  dated  so  we  were  not  able  to 
ascertain  whether  the  IRR  was  completed  before  the  draft  report  was  issued.  Also,  the 
referencer  did  not  indicate  any  concerns  during  the  IRR;  however,  we  found 
discrepancies  with  the  references,  such  as  a  figure  cited  in  the  report  as  representing 
fourth  quarter  data  that  was  documented  in  the  working  papers  as  representing  the  first, 
second,  and  third  quarters  only. 

We  found  misleading  or  unsupported  information  in  the  final  reports  of  the  two  audits 
that  did  not  have  IRRs.  Although  none  of  these  inaccuracies  affected  the  overall 
conclusions  of  the  reports,  they  demonstrate  instances  that  an  IRR  might  have  alerted  the 
auditors  to  before  the  reports  were  issued  to  management.  Specifically,  we  found: 

•  A  figure  cited  in  the  report  as  “a  contract  was  established  in  April  2000  for  $42 1 .0 
million”;  however,  $421.0  million  was  the  amount  of  the  contract  after  several 
contract  modifications;  the  amount  of  the  contract  at  the  date  of  establishment  was 
$402.0  million. 
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•  A  figure  cited  in  the  report  as  $330,000  was  documented  in  the  working  papers  as 
$300,000. 

•  A  figure  cited  in  the  report  as  52  percent  was  documented  in  the  working  papers  as 
53  percent. 

Over  time  if  errors  continue  to  be  found  in  reports,  the  report  recipients  may  begin  to 
question  significant  conclusions  of  the  reports.  AFAAI  65-101  does  not  exclude  the  1RR 
from  the  SAP  audits  and  does  not  define  circumstances  in  which  an  1RR  need  not  be 
done. 

Quality  Assurance.  GAS  3.50  requires  than  an  audit  organizations  internal  quality 
control  system  should  include  procedures  for  monitoring,  on  an  ongoing  basis,  whether 
the  policies  and  procedures  related  to  the  standards  are  suitably  designed  and  are  being 
effectively  applied.  This  is  often  referred  to  as  an  internal  quality  assurance  program. 

AFAAI  65-105,  “Internal  Quality  Control  Review  Program,”  provides  guidance  on  the 
AFAA  internal  quality  control  program,  and  notes  that  it  consists  of  four  components: 
supervision,  independent  referencing,  internal  quality  control  reviews,  and  external 
quality  control  reviews.  AFAA  conducts  three  types  of  internal  reviews:  functional, 
management,  and  quality  assurance.  Quality  assurance  reviews  evaluate  whether  AFAA 
activities  adhere  to  applicable  government  auditing  standards  and  internal  policies  and 
procedures  for  conducting  audit  projects.  The  AFAA  internal  quality  control  review 
program  uses  a  team  approach  to  accomplish  this  type  review  and  a  checklist  based  on 
GAS  and  AFAA  policies  and  procedures.  The  AFAA  Policy,  Oversight,  and  Systems 
Division  develops  an  annual  review  plan  that  describes  the  functional,  management,  and 
quality  assurance  reviews  planned  for  the  next  year.  The  AFAA  Policy,  Oversight,  and 
Systems  Division  also  prepares  a  3-year  review  plan  for  quality  assurance  reviews  which 
ensures  each  AFAA  audit  division/region  receives  a  quality  assurance  review  at  least 
once  during  the  3-year  cycle. 

AFAAI  65-105  does  not  exempt  or  waive  SAP  audits.  However,  AFAA  personnel 
involved  with  the  internal  quality  control  review  program  stated  that  they  do  not  conduct 
quality  assurance  reviews  of  SAP  audits.  AFAA  has  not  conducted  a  quality  assurance 
review  of  a  SAP  audit  in  at  least  10  years,  specifically  due  to  the  limitations  on  access 
because  of  security  requirements. 

Conclusion.  SAP  audits  seldom  get  the  monitoring  or  oversight  of  non-SAP  audits 
because  of  the  security  requirements.  AFAA  guidance  on  the  quality  control  system  does 
not  specifically  address  SAP  audits.  The  AFAA  Chief,  Special  Program  Division  has 
stated  that  he  is  unable  to  fully  implement  key  elements  of  the  AFAA  quality  control 
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system  because  of  security-related  limitations.  We  recognize  the  challenges  that  result 
from  these  limitations;  however,  AFAA  needs  to  re-emphasize  the  importance  of 
complying  with  the  key  elements  of  the  quality  control  system. 

Although  we  did  not  find  that  any  of  the  deficiencies  we  noted  would  individually  or 
collectively  justify  an  opinion  other  than  unqualified,  AFAA  needs  to  ensure  on  an 
ongoing  basis  that  their  internal  quality  control  system  within  the  SAP  environment  taken 
as  a  whole  still  provides  reasonable  assurance  that  GAS  is  met. 

Recommendations.  We  recommend  that  the  Air  Force  Auditor  General: 

1 .  Stress  to  SAP  auditors  the  importance  of  complying  with  existing  guidance  and 
procedures  related  to  independence,  specifically  in  completing  independence 
statements. 

Management  Comments.  The  AFAA  concurred  in  principle  with  the 
recommendation  and  stated  that  while  they  agree  improvements  were  needed 
to  document  their  independence  on  each  project,  the  AFAA  does  not  believe 
recommending  a  specific  software  package  was  inappropriate.  The  AFAA 
stated  that  the  audit  evaluated  various  processes  used  at  several  locations  and 
recommended  streamlining  and  improving  the  entire  operation.  At  the 
Command-level,  the  newly  developed  software  package  allowed  the  auditor  to 
easily  reconcile  total  cash  accountability.  However,  reconciliation  was 
difficult  and  labor  intensive  at  subordinate  units  because  the  tracking  systems 
were  not  as  effective.  As  a  result,  the  AFAA  report  stated  that  the  units  used 
different  procedures  and  recommended  the  Command  direct  use  of  the 
Command-level  software  package.  While  working  paper  documentation  could 
have  been  better,  the  AFAA  stated  that  they  complied  with  GAS  3.14  and  had 
sufficient  evidence  to  justify  recommending  the  Command-level  system.  The 
Special  Programs  Division  Chief  will  issue  a  memorandum  by 
September  30,  2005,  to  all  SAP  auditors  stressing  the  importance  of  complying 
with  guidance  related  to  independence,  including  preparing  independence 
statements  and  providing  advice  to  management. 

Reviewer  Response.  Management  comments  meet  the  intent  of  the 
recommendation.  We  revised  the  report  including  this  recommendation  based 
on  comments  to  the  draft  report  related  to  circumstances  in  which  the  auditor 
provided  advice  to  the  command.  No  further  comments  are  required. 
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2.  Ensure  that  the  SAP  auditors  and  supervisors  are  timely  made  aware  of 
changes  in  policies,  practices  and  guidance. 

Management  Comments.  The  AFAA  concurred  with  the  recommendation 
and  stated  that  the  Special  Programs  Division  Chief  will  issue  a  memorandum 
by  September  30,  2005,  directing  all  SAP  auditors  to  periodically  review  the 
AFAA  web  site  to  identify  changes  in  policies,  practices,  and  guidance. 

Reviewer  Response.  Management  comments  are  responsive. 

3.  Re-emphasize  to  SAP  auditors  and  supervisors  the  importance  of  complying 
with  the  key  elements  of  the  AFAA  quality  control  system,  specifically  related 
to  audit  planning,  supervisory  reviews,  and  independent  reference  reviews. 

Management  Comments.  The  AFAA  concurred  with  the  recommendation 
and  stated  that  for  audit  planning  the  Special  Programs  Division  Chief  will 
issue  a  memorandum  by  September  30,  2005,  to  all  SAP  auditors  emphasizing 
the  importance  of  complying  with  planning  guidance  contained  in 
AFAAI  65-101.  Regarding  supervision,  the  AFAA  stated  that  the  Special 
Programs  Division  Chief  will  make  working  paper  reviews  a  higher  priority 
and  will  delegate  supervisory  responsibilities  to  a  GS-14  for  some  projects.  In 
addition,  the  Division  Chief  will  issue  a  memorandum  by  September  30,  2005, 
to  all  SAP  auditors  directing  them  to  submit  periodic  project  status  reports  and 
document  project  related  telephone  conversations  with  the  Division  Chief. 

For  independent  referencing,  although  very  important,  the  AFAA  stated  that 
performing  independent  referencing  on  every  Special  Programs  Division 
project  may  not  be  practical  for  reasons  such  as  security  considerations  and 
most  locations  have  only  one  auditor  and  temporary  duty  travel  may  not  be 
cost-effective.  However,  the  AFAA  will  revise  guidance  by 
September  30,  2005,  to  give  the  Division  Chief  the  flexibility  to  decide  which 
projects  warrant  independent  referencing  depending  on  project  complexity, 
working  paper  location,  and  security  classification.  If  the  Division  Chief 
decides  independent  referencing  is  not  warranted,  a  memorandum  for  record 
supporting  the  decision  will  be  required.  Also,  independent  referencing  on  all 
projects  at  locations  with  more  than  one  auditor  will  be  required. 

Reviewer  Response.  Management  comments  meet  the  intent  of 
recommendation. 
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4.  Establish  as  part  of  its  internal  quality  control  review  program,  internal  quality 
assurance  reviews  of  selected  SAP  audits  in  order  to  comply  with  GAS  3.50. 


Management  Comments.  The  AFAA  concurred  in  principle  with  the  intent 
of  the  recommendation  and  stated  while  they  agree  including  SAP  audits  in 
quality  assurance  reviews  would  add  quality  control,  potential  benefits  must  be 
weighed  against  security  implications.  As  an  alternative,  the  Division  Chief 
will  have  senior  personnel  perform  complete  reviews  of  selected  projects  using 
AFAA  quality  assurance  review  checklists  and  report  results  to  the  Division 
Chief  and  Auditor  General. 

Reviewer  Response.  Management  comments  meet  the  intent  of 
recommendation.  We  recognize  that  the  SAP  audits  could  not  necessarily  be 
treated  the  same  as  non-SAP  audits  in  the  internal  quality  assurance  program. 
The  AFAA  alternative  procedures  meet  the  intent  of  GAS  for  on-going 
monitoring. 
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Appendix  C.  Management  Comments 


DEPARTMENT  OF  THE  AIR  FORCE 


AIR  FORCE  AUDIT  AGENCY 


9  August  2005 


MEMORANDUM  FOR  ASSISTANT  INSPECTOR  GENERAL.  AUDIT  POLICY  AND 


OVERSIGHT 

ATTN:  MS.  PATRICIA  BRANNIN 


FROM:  HQ  AFAA  DO 

1126  Air  Force  Pentagon 
Washington  DC  20330-1126 

SUBJECT:  Response  to  20  July  2005  Draft  Report.  Quality  Control  Review  of  the  Air  Force 
Audit  Agency's  Special  Access  Program  Audits 

My  staff  and  I  reviewed  the  subject  draft  review  and  are  pleased  you  concluded  our 
quality  control  system  meets  standards  established  by  the  President's  Council  on  Integrity  and 
Efficiency  and  provides  reasonable  assurance  Air  Force  Audit  Agency  personnel  are  complying 
with  professional  auditing  standards  when  conductmg  work  in  special  access  programs.  We 
concur  with  the  review  results  and  recommendations  and  provide  the  attached  responses. 

We  express  our  thanks  to  your  staff  who  conducted  this  review  m  a  professional  and 
effective  manner.  Should  you  have  questions,  please  call  me  at  (703)  696-7764  or  have  your 
staff  contact  Mr.  Jim  Sommer  at  (703)  696-7727  or  Mr.  Andy  Giancola  at  (703)  602-0172. 


MICHAEL  V.  BARBINO 


Assistant  Deputy  Auditor  General 


Attachment: 

Recommendation  Responses 
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Final  Report 
Reference 


Revised  at 
Pg-  11 


Revised 
Report  at 
Pg-  6 


RESPONSES  TO  RECOMMENDATIONS  IN 
20  JULY  2005  DRAFT  REPORT.  QUALITY  CONTROL  REVIEW 
AIR  FORCE  AUDIT  AGENCY'S  SPECIAL  ACCESS  PROGRAM  AUDITS 

1.  Recommendations.  The  An  Force  Auditor  General  should: 

a.  Recommendation  1.  Stress  to  Special  Access  Program  (SAP)  auditors  the  importance 
of  complying  with  existmg  guidance  and  procedures  related  to  mdependence,  specifically  in 
completing  the  mdependence  statements  and  ensuring  that  they  make  clear  to  management  their 
role  when  providing  advice. 

b.  Recommendation  2.  Ensure  that  die  SAP  auditors  and  supervisors  are  timely  made 
aware  of  changes  in  policies,  practices,  and  guidance. 

c.  Recommendation  3.  Reemphasize  to  SAP  auditors  and  supervisors  die  importance  of 
complying  with  the  key  elements  of  the  AFAA  quality  control  system,  specifically  related  to 
audit  planning,  supervisory  reviews,  and  mdependence  reviews. 

d.  Recommendation  4.  Establish  as  part  of  its  mtemal  quality'  control  review  program, 
internal  quality  assurance  reviews  of  selected  SAP  audits  in  order  to  comply  with  GAS  3.50. 

2.  AFAA  Comments.  We  concur  with  the  observations  and  recommendations  and  provide  the 
followmg  comments: 

a.  Recommendation  1.  Concur  in  principle. 

(1)  We  agree  improvements  are  needed  to  document  our  mdependence  on  each 
project,  but  do  not  believe  recommending  a  specific  software  package  was  inappropriate.  The 
audit  evaluated  various  processes  used  at  several  locations  and  recommended  streamlinmg  and 
improving  the  entire  operation.  The  “software  packages”  cited  were  locally  developed  Access 
database  applications  at  two  units  and  an  Excel  spreadsheet  application  at  the  third  unit.  At 
Command-level,  the  newly  developed  Access  system  allowed  the  auditor  to  easily  reconcile  total 
cash  accountability.  However,  reconciliation  was  difficult  and  labor-intensive  at  subordmate 
units  because  tracking  systems  were  not  as  effective.  Therefore,  our  report  stated  the  three  units 
used  different  procedures  and  recommended  the  Command  direct  use  of  the  Command  Access 
database.  Die  Command  concurred  and  has  used  the  system  successfully  for  two  years.  While 
working  paper  documentation  could  have  been  better,  we  complied  with  GAS  3.14  and  had 
sufficient  evidence  to  justify  recommending  the  Command  system. 

(2)  The  Special  Programs  Division  Chief  will  issue  a  memorandum  to  all  SAP 
auditors  stressmg  the  nnportance  of  complymg  with  guidance  related  to  independence,  including 
preparing  mdependence  statements  and  providing  advice  to  management.  Estnnated  Completion 
Date:  30  September  2005. 
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b  Recommendation  2.  Concur.  The  Special  Programs  Division  Chief  will  issue  a 
memorandum  directing  all  SAP  auditors  to  periodically  review  the  AFAA  web  site  to  identify 
changes  m  policies,  practices,  and  guidance.  Estimated  completion  date:  30  September  2005. 

c  Recommendation  3.  Concur.  The  Special  Programs  Division  has  several  unique 
challenges  in  these  areas. 

(1)  Planning.  The  Special  Programs  Division  Chief  will  issue  a  memorandum  to 
all  SAP  auditors  emphasizing  the  importance  of  complying  with  planning  guidance  contained  in 
AFAAI  65-101.  Estimated  Completion  Date:  30  September  2005. 

(2)  Supervision.  The  Special  Programs  Division  Chief  directly  supervises  eight 
auditors  assigned  to  six  locations  with  limited  electronic  communication  capability  for  security 
reasons.  This  situation  makes  it  very  difficult  to  review  working  papers  more  than  once  a  quarter 
in  most  instances.  As  the  report  mentions,  this  condition  is  mitigated  because  the  Division  has 
experienced  auditors.  The  Division  Chief  will  make  working  paper  reviews  a  higher  priority  and 
will  delegate  supervisory  responsibilities  to  a  GS-14  for  some  projects.  Further,  the  Division 
Chief  will  issue  a  memorandum  to  all  SAP  auditors  directing  them  to  submit  periodic  project 
status  reports  and  document  frequent  project-related  telephone  conversations  with  the  Division 
Chief.  Estimated  Completion  Date:  30  September  2005. 

(3)  Independent  Referencmg.  Although  very  important,  performing  mdependent 
referencing  on  every  Special  Programs  Division  project  may  not  be  practical  for  reasons  such  as: 
(a)  most  locations  only  have  one  auditor  and  frequent  temporary  duty  travel  may  not  be  cost- 
effective;  (b)  mailing  working  papers  to  another  location  is  not  a  viable  option  for  security 
reasons;  and  (c)  increased  auditor  access  to  sensitive  programs  is  not  prudent.  Nevertheless,  we 
recognize  the  need  for  action  and  will  revise  Agency  guidance  to  give  the  Division  Chief  the 
flexibility  to  decide  which  projects  warrant  mdependent  referencmg  depending  on  project 
complexify',  working  paper  location,  security  classification,  etc.  If  the  Division  Chief  decides 
independent  referencmg  is  not  warranted,  we  will  require  a  memorandum  for  record  supporting 
this  decision.  Further,  we  will  require  mdependent  referencing  on  all  projects  at  locations  with 
more  than  one  auditor.  Estimated  Completion  Date:  30  September  2005. 

d.  Recommendation  4.  Concur  in  prmciple.  While  we  agree  including  SAP  audits  m 
quality  assurance  reviews  would  add  quality  control,  potential  benefits  must  be  weighed  against 
securin'  implications.  DoD  Directive  0-5205.7  states  the  number  of  persons  granted  access  to  a 
SAP  should  be  strictly  limited  to  the  minimum  necessary  to  execute  the  program,  and  to  those 
who  will  clearly  and  materially  contribute  to  the  program.  Further,  the  directive  states  DoD 
Components  assigned  oversight  duties  should  perform  these  functions  using  small  cadres  of 
appropriately  cleared  and  accessed  personnel  whose  primary  responsibility  is  to  support  SAPs. 
Deviatmg  from  this  policy  to  grant  more  individuals  SAP  access  increases  the  securin’  risk.  As 
an  alternative,  the  Division  Chief  will  periodically  have  senior  AGS  personnel  perform  complete 
reviews  of  selected  projects  usmg  AFAA  quality  assurance  review  checklists  and  report  results 
to  the  Division  Chief  and  Auditor  General.  This  new  control,  combined  with  increased  Division 
Chief  oversight,  selective  independent  referencing,  and  peer  reviews,  will  provide  assurance  we 
comply  with  government  auditmg  standards.  Estimated  Completion  Date:  30  September  2005 
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